Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added several plan building speed improvements #5

Merged
merged 4 commits into from
Jan 5, 2021

Conversation

daniel-shields
Copy link
Collaborator

No description provided.

@@ -30,6 +30,7 @@ def install_with_constraints(session, *args, **kwargs):
"poetry",
"export",
"--dev",
"--without-hashes",
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the correct action based on python-poetry/poetry#3472 (search "without-hashes")

@@ -1,258 +1,250 @@
[[package]]
category = "dev"
description = "A configurable sidebar-enabled Sphinx theme"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seemingly just bumps a bunch of versions...

@codecov-io
Copy link

codecov-io commented Jan 5, 2021

Codecov Report

Merging #5 (ca114ad) into main (fabb587) will increase coverage by 0.05%.
The diff coverage is 97.61%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main       #5      +/-   ##
==========================================
+ Coverage   92.77%   92.82%   +0.05%     
==========================================
  Files          44       44              
  Lines        1619     1631      +12     
  Branches      230      226       -4     
==========================================
+ Hits         1502     1514      +12     
  Misses         94       94              
  Partials       23       23              
Impacted Files Coverage Δ
src/uberjob/progress/__init__.py 100.00% <ø> (ø)
src/uberjob/progress/_html_progress_observer.py 98.18% <ø> (ø)
src/uberjob/_util/validation.py 85.71% <66.66%> (+1.09%) ⬆️
src/uberjob/_execution/run_physical.py 98.36% <90.00%> (+0.05%) ⬆️
src/uberjob/_plan.py 100.00% <100.00%> (ø)
src/uberjob/_run.py 100.00% <100.00%> (ø)
src/uberjob/_transformations/caching.py 98.91% <100.00%> (-0.03%) ⬇️
src/uberjob/_transformations/pruning.py 100.00% <100.00%> (ø)
src/uberjob/_util/__init__.py 91.83% <100.00%> (+0.34%) ⬆️
src/uberjob/progress/_console_progress_observer.py 97.50% <100.00%> (ø)
... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 623b9ac...ca114ad. Read the comment docs.

@@ -88,24 +92,24 @@ def prep_run_physical(
plan, output_node
)
plan = prune_source_literals(plan, inplace=inplace)
on_started = on_started or identity
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This avoids three null checks for each node...

raise
finally:
bound_call.value = None
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving this to the finally block fixes a memory leak related to failures. In practice this probably didn't matter.

@@ -55,7 +55,7 @@ def _create_bound_call_lookup_and_output_slot(
plan: Plan, output_node: Optional[Node] = None
):
result_lookup = {
node: Slot(node.value if type(node) is Literal else None)
node: node if type(node) is Literal else Slot(None)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a tiny step towards eliminating Slot from run_physical entirely, which is possible but I don't want to do as part of this PR.

@@ -32,6 +32,7 @@ isort = "^5.5.3"
pytest = "^5.2"
pytest-cov = "^2.10.1"
releases = "^1.6.3"
six = "^1.15.0"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This addresses an issue with CI.

@timothy-shields timothy-shields merged commit cf424cd into main Jan 5, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants